Improved physiological noise regression in fNIRS: a multimodal extension of the General Linear Model using temporally embedded Canonical Correlation Analysis

For the robust estimation of evoked brain activity from functional Near-Infrared Spectroscopy (fNIRS) signals, it is crucial to reduce nuisance signals from systemic physiology and motion. The current best practice incorporates short-separation (SS) fNIRS measurements as regressors in a General Linear Model (GLM). However, several challenging signal characteristics such as non-instantaneous and non-constant coupling are not yet addressed by this approach and additional auxiliary signals are not optimally exploited. We have recently introduced a new methodological framework for the unsupervised multivariate analysis of fNIRS signals using Blind Source Separation (BSS) methods. Building onto the framework, in this manuscript we show how to incorporate the advantages of regularized temporally embedded Canonical Correlation Analysis (tCCA) into the supervised GLM. This approach allows flexible integration of any number of auxiliary modalities and signals. We provide guidance for the selection of optimal parameters and auxiliary signals for the proposed GLM extension. Its performance in the recovery of evoked HRFs is then evaluated using both simulated ground truth data and real experimental data and compared with the GLM with short-separation regression. Our results show that the GLM with tCCA significantly improves upon the current best practice, yielding significantly better results across all applied metrics: Correlation (HbO max. +45%), Root Mean Squared Error (HbO max. -55%), F-Score (HbO up to 3.25-fold) and p-value as well as power spectral density of the noise floor. The proposed method can be incorporated into the GLM in an easily applicable way that flexibly combines any available auxiliary signals into optimal nuisance regressors. This work has potential significance both for conventional neuroscientific fNIRS experiments as well as for emerging applications of fNIRS in everyday environments, medicine and BCI, where high Contrast to Noise Ratio is of importance for single trial analysis.Published versio

Contribution of speckle noise in near-infrared spectroscopy measurements

Near-infrared spectroscopy (NIRS) is widely used in biomedical optics with applications ranging from basic science, such as in functional neuroimaging, to clinical, as in pulse oximetry. Despite the relatively low absorption of tissue in the near-infrared, there is still a significant amount of optical attenuation produced by the highly scattering nature of tissue. Because of this, designers of NIRS systems have to balance source optical power and source–detector separation to maximize the signal-to-noise ratio (SNR). However, theoretical estimations of SNR neglect the effects of speckle. Speckle manifests as fluctuations of the optical power received at the detector. These fluctuations are caused by interference of the multiple random paths taken by photons in tissue. We present a model for the NIRS SNR that includes the effects of speckle. We performed experimental validations with a NIRS system to show that it agrees with our model. Additionally, we performed computer simulations based on the model to estimate the contribution of speckle noise for different collection areas and source–detector separations. We show that at short source–detector separation, speckle contributes most of the noise when using long coherence length sources. Considering this additional noise is especially important for hybrid applications that use NIRS and speckle contrast simultaneously, such as in diffuse correlation spectroscopy.R01 EB025145 - NIBIB NIH HHS; R24 NS104096 - NINDS NIH HHSPublished versio

Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification

In this paper, we briefly introduce the solution of our team HFUT-VUT for the Micros-gesture Classification in the MiGA challenge at IJCAI 2023. The micro-gesture classification task aims at recognizing the action category of a given video based on the skeleton data. For this task, we propose a 3D-CNNs-based micro-gesture recognition network, which incorporates a skeletal and semantic embedding loss to improve action classification performance. Finally, we rank 1st in the Micro-gesture Classification Challenge, surpassing the second-place team in terms of Top-1 accuracy by 1.10%.Comment: 1st Place in Micro-gesture Classification sub-challenge in MiGA at IJCAI-202

Cross-modal and Cross-domain Knowledge Transfer for Label-free 3D Segmentation

Current state-of-the-art point cloud-based perception methods usually rely on large-scale labeled data, which requires expensive manual annotations. A natural option is to explore the unsupervised methodology for 3D perception tasks. However, such methods often face substantial performance-drop difficulties. Fortunately, we found that there exist amounts of image-based datasets and an alternative can be proposed, i.e., transferring the knowledge in the 2D images to 3D point clouds. Specifically, we propose a novel approach for the challenging cross-modal and cross-domain adaptation task by fully exploring the relationship between images and point clouds and designing effective feature alignment strategies. Without any 3D labels, our method achieves state-of-the-art performance for 3D point cloud semantic segmentation on SemanticKITTI by using the knowledge of KITTI360 and GTA5, compared to existing unsupervised and weakly-supervised baselines.Comment: 12 pages,4 figures,accepte

BGM: Building a Dynamic Guidance Map without Visual Images for Trajectory Prediction

Visual images usually contain the informative context of the environment, thereby helping to predict agents' behaviors. However, they hardly impose the dynamic effects on agents' actual behaviors due to the respectively fixed semantics. To solve this problem, we propose a deterministic model named BGM to construct a guidance map to represent the dynamic semantics, which circumvents to use visual images for each agent to reflect the difference of activities in different periods. We first record all agents' activities in the scene within a period close to the current to construct a guidance map and then feed it to a Context CNN to obtain their context features. We adopt a Historical Trajectory Encoder to extract the trajectory features and then combine them with the context feature as the input of the social energy based trajectory decoder, thus obtaining the prediction that meets the social rules. Experiments demonstrate that BGM achieves state-of-the-art prediction accuracy on the two widely used ETH and UCY datasets and handles more complex scenarios